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Impiemeniation of the Entry Level Skills Program v 
(ELSP) during 1979-8* at Huston-Tillotson College and Wiley College, 
both historically black institutions in Texas, is" described. The 
ELSP, which is a developmental program fop freshman students who have 
not acquired the full range or level of cognitive skills needed for 
college-, is pact of -the Developmental Research Project, which is 
designed to provide an improved learning system, a management" 
information system, an appropriate test package, 'arid a monitoring and 
evaluation system. Overall, the project' was designed to provide* 
professionals, and particulajry women and minorities at the 
postdoctoral and predoctoral levels, the, opportunity to design and 0 
implement the evaluation research component .of the ELSP. The 
competency-based skills program is based on the planning, management, 
and evaluation w *systems concept to, instruction. In addition, the ~ 
program addresses testing concerns, and the study design includes 
administration of a standard i zed jpre- and posttest -battery and a 
series of criterion-referenced tests (e.g., the Comparative Guidance 
and Placement Program bat.tery). The Institute f-or Services to • 
Education Demographic/Psychosocial Scale and the Faculty and 
Administrator Survey form were also administered.- Information is 
provided on student and*f acuity characteristics at the two pilot 
schools, student performance on reading ,and /written* English 
expression, performance on criterion-referenced tests in English and ' 
biology, results qf item analyses of mastery tests, and results of 
faculty critiques of. the programs , in math and biology. Brief 
recommendations regarding the English tests (including reexamining 
items on the mastery test)*, the study survey guide, reporting forms, 
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INTRODUCTION 



, The Entry Level Skills Program (ELSP) is a developmental 

program for freshman students who have "hot acquired the full raftge 
or 16vel of cognitive skills required for^ a successful postsecondary 
educational Experience. The prograpi is in its exploratory, stages. 
The present report is the .second review and analysis in cbllabor- 
ation with two historically .black institutions — Huston~TiPl ; otson 

0 * » 

and Wiley Colleges" (Texas) . The .developmental study is entitled 

• 1 •* 1 a 

* * 1 « 

The ELSP Developmental Research Project (DRP) . ' DRP is ^designed 
to generate data regarding the following ^critical features 'of ISE's ' 



r 

i' ■ 



Competency, based elementary level skills program (C-B^LS) : 



* ; • goals and objectives - ' 

\ i 

• curriculum materials^ 

*• instructional strategies *' * ' : 

4. data collection, analysis, and reporting systems 

% • student*' performance ^ \ * j / -\ 

#. criterion-ref ererided test batteries 

other* instrumentation 
The present progress report covers the period August ^1979-% 
July 1980, and describes the procedures, and results from the iin- 
plementation of the program. A section of s the report is devoted 
tcuan .examination, of the various problems encountered 4 during the 



year and recommendations for implementation of the program in 
Academic year 1980-81. • '< % C 

DEVELOPMENTAL RESEARCH PROJECT PROCEDURES 




The methodology used , in the conduct of the DRP is the case 
•study; The design also includes the administration of the Compar- 
ative Guidance and Placement test battery as a pre- andposttest 
of student achievement, and a series of criterion referenced tests 

* * 

for diagnosis and* prescription within each discipline. 

Implicit in the nature of the pilot study is a two-pronged 
approach to t implementation: a Local Development Research Effort 
and a Cross Institutional .Research Effort. , * 

The Local Development Research Effort is designed to support 
the primary objective of ELSP, which 'is to facilitate instructional 

4 V 

decision making in the classroom. The activities include facilita- 
ting and dc>qumenting a review and critique of the following critical 
program features : ft \ % , ^ 
, . • Criterion Referenced Tests 

• Instructional materials (student and faculty) 

• Management Information System ' . 

• Administrative Structure 

The Cross-Institjrfc^onal Research Effort refers . to the ad- 
ministration of the standardized battery and 'to the following^ 
survey instruments which - provide demoographic and psychosocial 
information on students faculty and administrators: .' - 



" . • The ISE Student Asselsment V- 

• Faculty and Administrators ' Attitude" Survey !' 
/ S tudent Course "Evaluation • , , 

- An exploratory study in nature, the DRP is designed to 
^ provide, ox^a reiterative basis, guidance for producing an improved 
^ELSP with proven validity. * , 7 \! 

»**, ' » • f 

- Expected Outcomes „ \ . * v , 

. *he primary expected.,outcome ' of the DRP .is "an improved . s ' 

and expanded learning system which consists of a teacher' s 'manual, 

syllabus, s/udent manual for .each of the .discipline areas-, a 

management information system; and -appropriate test package and 

a monitoring and evaluation system. x It is expected that' this 

system will eventually be implemented in schools with populations 
." V ' * ' < • 

.-similar to that of the pilot schools. 



Population &nd Sample 



The target population of the. disadva'irtagek • students in 
..approximately 20 of ^ the nation's .105 historically black- colleges,. 

0 ♦ 

The terminology- "historically black institutions" refers to "those 
institutions of higher education' that were founded primarily for 

black Americans, Negroes or Colored TPeople, 'although their charter 1 

« " - '* • . . 

were, in most instances, not exclusionary"' (Blake, ,et al, 1975)". 

More specifically, most of these institutions have served black 

'* * • • * * 

American's for at least two decades arft. most are frdm 50 to 100 , 

. • , • ' ** 

years'old. . . . , » • * • . / 



As nWtion^d previously the pilot study sample consists 

of two historically Black colleges; ^Huston-Tilltftson ' and Wiley, 

both in Texas. Criteria for selection of these institutions - 

included: *" administrative, and faculty commitment to the ELSP 

V 

Program, four .year college .status and location in the. south. 
Methodology . 1 * * - 

:The methodology, ^sed in .the conduct of the' pilot study 

was the case study approach. Data was collected through a mix 

of observations and, interviews., "in' addition>*t£e design also ' 

included the administration of a standardised/pre- and posttest 

battery and- a , series of criterion-referenced tests.. 

Implementation , Standardized guidelines were established and 

implemented to insure. a core of* commonality across the institutional 

sample. Guidelines aire as follows:* , • 

« * , • • • 

• Establishment of a local task forcq which includes 
the ELSP director, faculty representatives and key 
administrative officials.. ' . 9 * < 



Focus on s verbal, analytical and critical thinking skills 
across* the discj^>linB areas. . .f 

Teaching of instructional objectives included, in. the-, 
discipline models. . f r*' 



# % Use of the ELSP package, including the syllabi, student 

manuals, recommendations regarding. using supplementary 
K materials and .recycling, and the instrumentation package. ' 

# Adherence to data collection'and , utilization and reporting 
guidelines; and « ' % * # • " * 

# Adherence to monitoring guidelines. * 



. -5- 



Participants^ wer'e urged, to^document systematically all other 
materials, techniques, tests "and procedures used. . * V 

Data Col lection Techniques . ' A variety of instruments were'* - 
used, to generate the data required' for validating the C-^LS 
program, including a faculty log, a standardized pretest-posttest 
battery, a. series of criterion-referenced tests, detailed inter- 
view schedules, checklists, and an institutional .inventory. 

' > .The faculty, log is the primary tool used for documenting 
experiences- with ELSP within each institution. Faculty. were asked 
to use 'this form oh a daily basis to- record in detail the following 
kincls of information: objectives; enabling activities and * 
materials, tests administered, including the. instructions given 
and the tinfe requirements; critical incidents observed; the 

results of the criterion-referenced tests (or other tests used) ; 

•* ' 8 • ■ • :f - v, • 5 ' 

as well as the utilization of such tests; turnaround time in 

.terms of toe receipt of materials and reports and recordings-; • 
changes i'n instructional plans; consultant visits and workshops ; 
and, attendance/attrition data. These logs were reviewed by DRP 
coordinators (arid the ISE Task Force) on a systematic basis and " ' 
become' the basis for change in curriculum materials'^ring the* 
Summer Conference. -» , . - ... 

Checklists were used by the D?P director and the ISETask '■»', 
Force in monitoring the project. .The purpose of these "checklists 
is 'to capture the progress which is being made 'in fdur treas' of ' 
the program: . instructional "management; student- information system 
development', including' date flow, file maintenance and dissemination; 
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research and administration*. • / * 

^ Two. types of instrumentation were used to obtain, performance 

and demographic data: a standardized pre- and posttest battery 

* • * 

consisting^ of the Comparative, Guidance and Placement Program 

battery, the ISE Demographic/Psychosocial Scale, and the Raculty 

/ 

and Administrator Survey Form, and qriterion-ref Srenced tests. 

The Comparative Guidance and' Placement Program battery 
consists of tests in the are.as of reading,, written expression, - 
computation, applied arithmetic and elementary algebra. The ISE 

■ a % 

Student Survey investigates the relationships between students' *•• 
self-concepts, their- feelings about themselves and their demo- 
graphic characteristics (e.g., race., age education and occupation, 
of parents). "The Faculty and Administrator Survey Form documents 
experiential backgrounds, responsibilities other than' .teaching, 
attitudes toward the C-BELS program; and attitudes toward dis- 
advantaged students. 0 " 



^ RESULTS OF THE 1979-198,0 PILOT STUDY 

Introduction , t 1 *. . " 

The results presented below emanate from the implementation , 
of .the ELSP' program is the two pilot study schools in the 1979-80 

i 

academic year. Since the two .schools presented* similar profiles, 
results to b^, analyzed .were randomly selected for each discipline 
area from one or another school. Also> rather -than examining the ' 
results 6f the whole freshman class in each discipline area, the 



\ 



\ 

results of a section were examined. A glance at the .overall 
results had indicated that all sections in each discipline area 
-appeared to have progressed in similar fashion in both- schools. 
In analyzing the resultsV three major research questions were 
posed/ 

^ 1. How valid is the ELSP Program, for £tS-^tud^it 
participants? ' 
2., How effective has the program been in terms of 
* , the following -fiwo outcomes? * 

v J » • 

(a) Overall gain in knowledge and skills in ' 
^ particular dis-cipline areas 
(\b) Overall gain in competency in taking v 
standardized tests ' 
3. .How reliable aire the test items and the tests 
themselves? 

Necessarily the interpretations of the results are tentative. 
There are several 'reasons for* this. 

ISE did not have the time or' the money to establish an exten- 

sive data barse which could accommodate all the information received 

from .the schools. Thus, no correlational or causal analysis could 

*« » 

» * * * 

be undertaken. * 

• . Second, the sheer extent of .the reporting system made it ; 
impossible for the schools £o keep to timeliness "and supply informa- 
tion in a timely fashion. * *. . * \ 

* * '* 

The flrst^ section of the\results presents an „ analysis of 

faculty and student characteristics. The second section will" 
present an analysis of the performance' Of participants after one 
academic year in the program. The last section will present an ^ 



. analysis of the reliability and validity of the cr'iterL referenced 
'tests' items and the tests themselves. *' ' 

FACULTY AND ADMI NISTRATORS • 
, The- faculty and administrators survey' was administered to all 
faculty and administrators in the two colleges who were directly 
or indirectly involved with the ELSP Program, 1979-1980. Since. the 
total .-number of such persons was only 18, 'the data from the' two 
schools were combined. The purpose of the surve^ttTprovide. the 
evaluation team of the ELSP Program with background information on ' 
the staff of the two institutions who were actively involved in 'the 
implementation of the pilot program. Questions concerning faculty. * 
attitudes and perceptions of the ELSP Program, teaching. experience, * 
■ experience with various teaching practices and research and ^valua- 
tion were presented to the -15 faculty members in the two institutions 
Our findings tfor tfie 1979-1980 academic year are as follows: " 

The median age of the'. respondents was 54.3 years, with 56% 
oyer 50 years old. Eighty-nine percent of the' respondents were ' . 
.black and 6% were white. ■ 

. Thirty-nine percent of the. respondents reported that they, held' 
terminal degrees, while 561 indicated; -that they held Master* degrees 
in either the sciences or the arts. Only onet respondent reported " ' 
not having an advanced degree. The majority of the respondents ' ' 
reported that they had had previous teaching experience at. the ' . 
college and; university level 1 83%. Only one reported no previousN 
work experience. ' > ' * * — 



> The present, faculty's involvement' with ELSP, indicates .that 
the. greater percentage, had participated in the program long enough 

• to. have a fair understanding of^its philosophy >nd- goals. Fifty- 
. three percent 'of the respondents reported more "than two years 
invplvement with, the -ELSP program* VThis fact was emphasized by 
the fact that 83% of the respondents^ had a fairly clear under- 
. standing of the process, philosophy Aid goals of. ELSP. . 

•There was some indication that 1 small but significant number 
of faculty were concerned about the increase in work ^load '-occasioned 
by the implementation of the Program; 2$%' of the respondents reported 
that the work load was excessive. There\ appeared to be a correla- 
tion between those, respondents who indicated _ that their participation 
in the program was, mandatory and those whoVelt that the^work load' 

.'was excessive. \* \ 

The faculty of both schools appeared toVave been greatly 
••involved in the planning and orientation se^sioi^s of the ELSP * .> 
Program. Seventy-eight percent of the respondenk>repqrted having 
participated in some form in the planning and o'rieWtion phases. 

* * \ \ * 8 

Of these, 83% participated in pre-Sumcner Confer ence\planning/at 
the institutional level, and '75% reported -participation at the ' « 
summer conference. The level of participation at postipsummer • 
conference was considerably, less at both ' ins td,tutionartnd ISE /' 
'personnel level. Only 11% participated at tire post-corif\rence 
planning at Dillferd, and 39% at post-conference planning ind '< 
, institutional level. Only , four respondents reported- that;Shey* 
liafl^not participated at any^conferences or meetings. 



Overall, the average faculty' member in the two institutions 
appears to. be' well oriented to the goals .and philosophy of the 
ELSP program and to be in agreement with most of them and with ' 
the process being used for' their implementation. However, there' 
appeared to- be* some disagreement over the' types of instructional 
materials used and with some of the procedures. ' * * 

In- general -the findings may be summarized as 'follows: 

• ELSP faculty and administrators are predominantly 
black. - I 

• 'almost all ELSP staff hold at least one advanced 

• degree-..- ' 

• Almost all ELSP faculty have -had previous teaching ' 
experience at the university/college leVel, and are 
experienced,, in development/remedial education, as - 

v well as- in some areas of curriculum design, e'valua- 
* tion and research. • ~ . • 

> • Most ELSP - faculty have participated in at least one 
of the orientation and planning seminars of the 
program. ' ■ 

• Finally, most ELSp faculty are oriented toward the 
philosophy and goals of the program,, have a clear 
understanding of them and somewhat agree, with both 
the goals and philosophy as well as the process- for 
their implementation. 

b. student' characteristics - r - 

• • The ISE assessment was administered to' all freshmen students 
at the two colleges. The >questionna'ire consists of 73 questions' 



designed td characterize the student .in terms of socio-economic 
background, self -concept variables and attitudes* toward various 
'academic subjects. * * 

The responses of 207 students (representing, the -tw/ schools) 
were analyzed. The results of this analysis are printed below-. 
. , On the whole, the student sample of *ELSP- appears very 
• similar in sex, age and racial composition to. the .population that, 
normally attends the Historically' Black College. The. group is "36 
percent male and 64 percent female. Blacks make up 9^ percent of 
t;he sample group, and the valance contains foreign students, 
American whites and other etnnic minorities. Only 5% of -the 
students are 21 years old -or older; the rest are between 16 and 20 
{See Table 1) . > 

A primary objective of ELSP is the reduction of attrition- 

* therefore, ' one section of "our analysis focused upon. factors 'which 

. • . \ - 

are, generally pivotal in -whether a student persists or drops out. 

• ■ • \ ■** * • 

One suph. factor is, of course-, financial- resources. In this 
<■ area, r .the C-BELS population did not fare well. While 44, percent ' 
.of the students reported ar family income of $15,01)0.00 or more, 
a significant 40 percent of 'the group reported their families 
incomes" to-be below $5*999.00 per, yearkich'is'at poverty level. 
Astin (1975) report? that ^his* findings and those of- others confirm 
a strong relationship between family . income ^and rates of attrition".' 
Table II shows that, on' the average, 31% of- the students who report 

V it 

.a family income below $4,000 are going to drop out. The" figures 
range from 24-29% for 'those in income categories between $4,000 
and' $15; 000. . ' * -wV ' ' 



TABLE 1 



COMPARISON OF 1979 - ; 80 • AND 1978 -79 ELSP STUDENTS 
, AT TWO 'PILOT. SCHOOLS ON SELECTED VARIABLES 



Variables' 1978-79 1979-80 

__: N = 368 N = 207. 



% ' ■ %• 



SEX 



' Male 49 ~ 36 

Female ,« . ' 51 v 64 

: v • •. , • 

< * 
AGE \ ' 

> 

16 -20- . 70 95 

Over. ,21 . • 30 5 . 

PARENTAL INCOME 

• 15,000 and above 24" "44^ 
Below 5,999 • * * 50 , ,'• 4£ 

PARENTAL OCC. . 

& Domestic and. ' 
UnskiTled labor ^ 55 «. 45 

Official and.. : ■ * 

Professional 10 • 13 



- * 



RACE 

< 



Bl&ck. * 85 - 90 

Other f 15 — c 10 



TABLE 2 

Family Income as RelateaYto Dropout Rates 



Less than 




4,000 


' 5,999 - 


6.000 - 


7 QQQ 


8,000 


9,999 " 


10,000 - 


14,999 - 


15,000 


19,000 


20,000 


24,000 


25,000- 


29,999 , 


30,000 or 


More 



• ,0 

Drop^Out Rates % 

31% 
" 29 

27 ' 
' '27 

24 

23 

16 



SOURCE: Astin, Alexander W. , Preventing Students' f remapping but 
j% Washington; Jossey - Bass Pub., i9?5, s p. 35. — 

comparison was made between the first participants in the 1 

ELSP pilot study (1*78-79) and the present participants, to 

determine if the assumptions under which the ELSP was designed ^ 

and implemented were valid for . the 'present* population. Table 3 * 

presents the results. The comparison indicated that there was 

great similarity between the. two groups. This appeared to 'indicate' 

that the ELSP. program was equally valid for. the present- participant; 

population, one section of the student assessment instrument . , • 

required students to rate themselves on, a number of issues/. This 

was in order to measure students » expectations and self-concepts 

both about .themselves, and their academic ability. The results. 

indicated that on the whole the ELSP participants had a realistic 
* -j . ' • * 

and positive self-concept of their ability and the skills which 

they had acquired, prior to entering \co liege/ 



items . 



Table 3 presents a 'summary of student responses to selected 



Table 3 



Selected affective characteristics of ELSP students 
l#79-&0 (N=207) 



Characteristics 

t A. • Ob^ectivfes Considered very important:/ * 
< * * 

be an authority in my field 

raise a family " 

be successful in business 

B. Student se*f rating on: 

(1) School achievement 

superior 
above 'average ~ • v 

average v 
. below average 
no response 

(2) Intelligence 

superior 
above average 
average 
below average 
ft no response 

(3) Qualitative and quantitative skills 

superior % * * 

above average 

averiage 

u below average - 0 ^ * • 

. „ • ~ , v no. response , * 



Percent 
30 

» 

5 

28 



5 
23 

• 

6? 
2 
3 

7 
31 
■ 57 
1 
* 4 

3 
•17 
66 
5 
9 
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C STUDENT PERFORMANCE ON ELSP 19 79-1980 ! - . 

CGP PRE AN D POST TEST (Reading, and Written English Expression) . 
The performance of 100 students in the reading and written 
english' expressions sections "of the CGP were examined. The CGP ' 
.was administered as a pretest at the beginning. of the academic 
year (August .1979) . .The post, test was administered at the end of 
the 'academic year (May 1979) , after two semesters' of; the ELSP. 
The results of the reading pretest indicated that*71 of the 100 
students scored at dr below the 20th percentile and 95 werj/below 
the 50th. Only 5 of the students scored above the 30th percentile. 
Of the 95 students who were at or below the 20th- percentile, 50 
were below the 10th*percentile. In other words, the pretest 
revealed that more than half the freshman class were, below the 10th ■ 
percentile in reading^skills-. This is particularly alarming when - 
•it is considered that the CGP is- geared to the 6th grade level and 
above -in reading' skills. Thus it can be estimated that 50% .of the 
freshmen class were reading at op below* 6th grade level. Post test 
CGP was administered at' the end of the^ school year and the following 
results were obtained for the same gispup of 100 students. There 
was an obvious improvement overall. As compared with 59 students 
who had been below the 10th percentile in the pretest, only 22 \ 
students Aow appeared", .to. be. below that level: .This was a signifi- 
cant gain. Even more significant was that 24 students now scored 
above the 50th percentile, when compared to only 5 in Hhe pretest. 
(Table 4). The students were then divided into low and high groups, 
a^d their pre and post test standard score examined. A, t test was 
performed to determine if there was a significant difference between 
the' pre-post-test means of each group. The results indicated "aK, 



dp?.*- -~ • • 

"ERIC ■ ..W.< • 16 
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significant difference: for the low group t=11.20, significant 
beyond d=-.001 dt 24 and for the high group t=1.62 significant- at 
^"=.1 dt 24.- It -was concluded that- the ELSP program had 'made "a 
difference. Participants appeared to have gained 3 grade years 
on the ~ average during the academic year. 

* . TABLE 4 

DISTRIBUTION OF PERCENTILE ' RANKS OF 1-0 6 

• V STUDENTS BEFORE AND AFTER TWO SEMESTERS • - 

« ' • OF ELSP PROGRAM . , - . ' ' ' 

" CGP READING. 







Percentile 
Rank 

* • 


Before ' 


After 








Above 50 • 


'• a- 


*24 "* 




; ^ 




41 - 50 


■ 4 


7 


* 


















31 - 40 , 


7 


15 • . 






W 


21 - 30 


• * 

9 . 










11 - 20 


16 


12 








1-10 


/ 59 


22 





i 



Similar analysis was conducted on the performance of the 
participants in the written English expression section of the 
CGP. Comparable results were obtained (Table 5) . 



TABLE *$ • * 

DISTRIBUTION OF PERCENTILES RANKS OF 97 STUDENTS 
BEFORE AND AFTER TWO SEMESTERS OF ELSP. PROGRAM 

* t 

CGP WRITTEN ENGLISH- EXPRESSION 



Percentile Before , ' After 

Rank • • * 



Above 50 
41 - 50 
31 - 40 
21 - 30 



.11 - 20 
1-10 



•"V 



12 


20 


2 


4 


5 


9 


13 


22 


16 


22 


49 


20 



. Seventy-five percent of the students' scored at N below the 
•^lOth percentile on th$ CGP administered before ELSP program/of 
i these, , only 22% scored" below the 10th percentile' on the postV 
test. An equally Significant difference was observed with those 
scoring above the'SOth percentile at the pretest only 5 of the 
100 students scored at thifj^evel. At the posti test, this number 
increased by .-19 to 24% • 

A similar division of the participants into low and high 
.groups was undertaken. The analysis of pre and post-test means 
indicated significant difference for the low group. No difference 

in pre and post- test performance was observed for 'the high group. 

i - . 



However, a slight improvement was indicated when an examination 
of individual scores was made. The .results again indicated*, 
that the ELSP appeared to have made a difference in the • 
performance of the participants (Tables 6 and 7) . 



TABLE 6 

v ; 



Distribution of Standard Scores of High Group* 
Before and After Two Semesters of ELSP.* N = 15 

CGP Written English Expression 



s 



Standard 
Score 


Before 


After 


56 - 6(T \ 


2 

. • o 


2 

• 7 


51.- 55 V 


9 ' 


1 


• 46-50 


3 


3 ' 


41 - 45 


1 

* 


2 



/ 



J Pretest X=52^.53 s=8.5 Posttest X=53^80 s«8.-5-3" 

. _ t=..95 df 14 

Performance on the Criterion Referenced Tests (CRT) 

-Before the prdgram was implemented, students were administered 
a CRT Mastery Test in eacn discipline area.' The same test was 
administered at the end of the program, to-' determine the extent to 
which the program *had led to an improvement in. tie skill level of 
the participants.. 

19 



TABLE 7 

I 

DISTRIBUTION OF STANDARD SCORES OF LQW* GROUP N=24 ON 
■ CCjP WRITTEN ENGLISH EXPRESSION BEFORE AND A*FTER 2 
\ SEMESTERS OF ELSP 



r 

» — 








'Standard Score 




Pretest 




* % 




•0 


2 


41-45 




0 


7 


^fi-4n • 




0 


7 


31135 . ' 




15 


7 


26-30 




6' 


1 


21-25 ~ 




3 


0 










. Pretest X = 29.96 




SD = 9.5 




Post test X = 38.29 




'*SD = 8. v 7 





t = 8.29 d = .001 dt 23 
A significant difference was observed in the performance of 
the low group between the" pretest before they received ELSP 
program and the post test. We can see that program made L 
some of the difference observed. "Before treatment, no 
student scored above 40 after treatment 9 or of the. students 
scored over *4ot 



D. ENGLfSH 



Table 8 is a : presentation of the Distribution of Scores 'on 
the Pre-Post Test English CRT Mastery. The scores are for 26 
students in a section of the ELSP. English program. A*t test was 



performed to* determine whether there *was a significant dif f e/ence* - « 
between* the pre^post-test means*. . The." results indicated that the - 
participants. appeared to have improved after two semesters of* . 

treatment t=4f292=.05. - • -* 

•' however ,^an examinat/Jn of the distribution of the pretest 
scores^ indicated that the scores, appeared to, be inflated and did ' ' 
not have any relationship yith the range of scores obtained by the 
participants in the CGP- Reading and\tfritten Expression sections. 
It is suggested that the CR^ items be closely examined tq determine 
whether they*' were- too easy and therefore not geared to. the level 
which the students were expected to attain aftei? treatment. * • . 1 



TABIjE 8 . • • 

" - ' ' • *• 

DISTRIBUTION OP STANDARD SCORES ON ENGLISH CRT MASTERY 
BEFORE ANt> AFTER' TWO SEMESTERS OF ELSP^ N ^g 1 ?*™**. 



Store ' Pretest, 



Post test 



86 -■ 90 . - » i 

-■81 - 85 i 

76 - 80 ' 3 

' 71 - 75 • •' 4 

66-70 8 

61 - 65 . 2 

56'- 60 * • . 3 

51' - 5S. , o 

' 46 - 50 3 

41 - 45 . . • . 0 

,.36 - 40 • * i 



2 

6 
5. 
5 
1 

3 
1 
0 

0 '* 



. 0 



Pretest. X='66,08 s=8.7 Post test 7=72.73 s=8 5 
• t=4.29* -d=.05 df 25 



y 21-i 



- BIOLOGY • ■ ' - • ' * ' * . 

* " 

The pre-post test scores of the* total, biology class were 

analyzed, in addition the class was divided into high and low 

groups. The Mgh group were those who scored' at one- standard 

deviation above the" pretest mean"; .the low group were those who 

^ scored less^han one Standard deviation. below .the .pretest mean. ' 

Table 9 presents the distribution of scores of. the -tdtal Biology * 

class.- The -i-.test indicated'; a Significant difference between 

pre-post test .means t=8.78 d=.001.. Subsequently l a latest 'performed 

^6h_the pre-post test means of the '.high- group indicated a difference 

at the .05 level t=1.94. (See Table io) .] : ' 



t 



* .TABLE" 9. " . ' 

' /*»«•.* , ' 

DISTRIBUTION^ OF SCORES ON THE BIOLOGY CRT MASTERY . 
OF 46 STUDENTS BEFORE AND AFTER 2 SEMESTERS OF ELSP. 



Score. 


PRETEST N 

1 , • 


POST TEST 


" 46-50 


1 

0 

< 


2 


Ul-45 m "* 
. 36-40 
31-35. 


1 

. * 1 


" > 12 
7 

• % V. 


26-30 
21-25 


' 6 %. • 
lit ' 


X 7 . 

• 7 


1 16-20/ 




• . 4 


11-15 • 


9 .1 

> • 
♦ « / 


« 

1 


6-10 

is * 


,« 


0 


4 Pretest X=21.43 
SD=7.9 • ' 
t=8* 78 beyond. .001 


Post^Test X-41.15 j. 
• SD=9V4 j 
(two tailed) H 

* * 
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» ! - .TABLE 10 

\ 1 ' . DISTRIBUTION OF SCORE OF HIGH GROUP 

* ' -ON MASTERY BEFORE AND. AFTER ELSP.. 

Nsl6> . HIGH GROUP 





* 








*■ 


Score 






-Post-: . 

t 




46-50. * ' 




- ■' ° * ' , 


t 

*■ 2 ' " • 


i 


41-45 




2- , 


3 J 




36-40 




1 


3 




• -31-35 ••' •• 




1 


3 




26-30 
21-25 




-<>6 


0 

■2 * 




16-20* 




• o * • 


i ■ 


V ' 


X=28.94 • 




' X = 33*56 * 






SD=9 . 48 


a 


\ SD=10 . 70° 


*• 




Max. score 


= 50 


» 




•4 


Min . score 


= 17 








t=1.9'4 . 


d=.05 


df=15 





The greatest gains appeared fco have been made by the low' group. The 
* t test yielded a tof 9.69 whiW was significant beyond the .001 • \ 
level. & - » 

The results of the Biology CRT- Mastery indicated that partici- 
. pants^had. greatly improved their knowledge and skills in Biology 
over the 2 semesters. It was therefore inferred that the significant 
charges observed were in part due to the ELSP Biology Program. 
(See Tabi'e 11) . 



' t ' ' TABLE 11 j 

DISTRIBUTION OP SCORES OP LOW GROUP ON BIOLOGY ^MASTERY 

BEPORE AND 'AFTER ELSP N=15 

V . 







• 


* 

V 


• Score 


Pre* 


• • 


Post 


41-45 . * 


• N 




♦ 

: * 6. 


36-40 


0 • 




2 


. ; ' - 31-35 \ • : 


-0 




'2 


26-3Q . • ' . » 

* 


0 


> 

• 


2 • 


21^5 


' o 




2 - 


V' '16-20 

« » 

11-15 


4 

9 

e 




1 


*- „ " 0 _ / 

-•6-10 / . 


2 




0 


-Max. score ? 50 


. — V 


• * o 

score = 7 




0 X =- 13.33 • 
' SD = 11.95 


X = 
SD = 


34.2 

9.98 : 




- t = 9^69 beyond 

* 


.001 two tailed 




« . - 









The results of^the CRT pre-post mastery tests together with 
the gains observed in the CGP. Pre-Post administration, -appear to 
indicate that the ELSP intervention program has contributed 
significantly to improving the knowledge^ and- skills level of the 
,partici^ants. However,, further Analysis of the data needs .to be 
carried, out to determine *how milch of the changes observed can.be 
attributed to the ELSP Program, itself. 



- ■ r \ 

Analysis of CRT Items . ' v 

* Since the ELSP Program is still in the experimental . stage , 
it is necessary that the tests "which are designed to measure student 
progress within the program be analyzed! to determine how va^lid'dnd 

CI 

reliable they ^re. In addition, j.t is essential that te3t items^bfe 

analyzed to determine their difficulty level and how well each item 

discriminates- between -the highest and lowest scores 

Ah item analysis was -carried out" on several items of the 
« 

Mathematics CRT Mastery to determine their difficulty level and 
their discriminatory po*S£rV \ * 

The results for 3 randomly selected group of items are 
presented, in Tabie 12. The results indicate for example that . 
dt;em i was too, difficult for all students and therefore did not' 
discriminate high scorers from low scorers. Item 15 is an example 
of 'a good item, in that difficulty level is about ,50 and ,the 
discriminating power of the' item is "about .60. Item 50 would appear 
to be a poor item; in that the difficulty" level was low .25, since 
the majority of both high and low scorers answered correctly and 

the discriminating power was also low .17 

r t 

Finally an attempt was made to determine how reliable the CRT 
Mastery tests were. The Kuder Richardson -test (k2r) was performed , 
on the pre and post test means of each CRT mastery. - The reliability 
coefficients ranged from .75 to .84. This indicated a high level 
of' test- reliability. 

* t 

Faculty Evaluation - 

The analysis of the faculty critiques of the program was 
limited to the* Math and Biology programs , since these yere the only 
two sets received by ISE at the time of analysis. . 



ITEM ANALYSIS OP RANDOMLY SELECTED. ITEMS 
ON TliE MATHEMATICS MASTERY- TEST** 
'.ON THE MATHEMATICS MASTERY* TEST 







Item No." / 


P • 


D 




> 














/ 


1 




1 ' * ' - 


•0 








% 




.4 • - 




• .67 










15 


'.50 


.66 , 










32 


. .58 . ' 


.16 : ' • 






1 




40 


■ ..50 
* 


is .• . . 

> .66 , • . 


6 








48 ' 
50 


.33 
.75 


• ' .66 
.17 







£ . The analysis- indicated that faculty were .pfeised with the ' 
manuals and other instructional tools provided by. ISE personnel.' 1 * 
They found that' on the whole the students progressed well under 
ELSP. Some concerns were 'voiced about the reading level 'of the, * ° 
manual, and about the^eed to expand practice exercises and include 
more information on supplementary materials. Faculty were especially 

•concerned with the amount of paperwork they were expected to . undertake 
during the course of implementations.. The majority 'suggested that' " 
the workload be reduced so that moije time--could be^- spent helping the*** 

•individual student. Overall, however-, faculty" were well satisf leS ' v 
with tha« ELSP program and found it both viable and effective'.' A * 
typical faculty coirafet was? "This program should have- been thought 

of , a long time -ago." ^ ; *.<f 



DISCISSION 





The goal of the present report, is to present a systematic 



thdugh limited « review of the processes, products, strengtll and 
weak\ejses. of the ELSP "(1979-1980) . ' " - - 

_ • The limited analysis of the ELSP Program 1979-80 presented - 

A here, -indicates that the program is progressing successfully, as 
far as its implementation at the institutional 1^1. Faculty 
appear to have followed the guidelines supplied and have routinely 
recorded all data 'from the pre-pbst test, CRT Mastery and^interim 

.tests for each discipline area. \ 

' \ L 

An analysis of the progress of the program 1979-80 "however , 
reveals certain problems, which need' to be ' examined if the program ~ 
is to become even more effective. , 

First, the amount of paperwork expected of the faculty is 
_burdensome. rfhere needs to be a greater simplification of the 
bookkeeping" 'process, which wili^hot endanger the formative aspect 
of the program. " " 

Second, a true assessment of the validity and effectives* 
of the. ELSP cannot be accomplished until a complete data -base is* 
set up. Without such a base, correlational and causal analysis 
cannot be undertaken. Such analysis is necessary to. allow, for a « 
better understanding of the relationship between tht^ entering ' 
characteristics of -the participants and their subsequent performance 



RECOMMENDATION S - *' 

1. The lack, of relationship between performance on the 
English CRT Mastery and the CGP Reading and Written English Sections 
requires that the items oh the Mastery test need to be reexamined. 



2. The student survey guide needs to be revised to eliminate 
the discrepancies in student responses to related items. For 
example , parents 1 occupation and parental income. 



JN^he reporting forms need to be streamlined N so th^t faculty 

« 

w^ll be required«to undertake less paperwork. 

- 4 « ISE must devise a. means to establish a computer data base 

so that more vigorous analysis of the data can be undertaken. 

5. No cut-off score has yet been arrived at aboye which , 
mastery can be assumed in each of the discipline area CRT mastery 
— tests. • It is suggested that analysis .of pre and post test scores 
for .the last two A ye,ars be undertaken, in order that a cut-off point 
can, be established, " based on a -relationship between pre-ppst test 
^ performance 'for the last two years. The establishment of cut-off 
scores— wiitiallow .for more effective placement of participants in 
the program, and for more individualization of instruction. 



